A video processing system captures or receives video frame data and perhaps other image data in processing video frames for a video sequence. The video frame data is formed by an array of pixels. Each pixel is defined by pixel data identifying specific color components, color characteristics, and/or grayscale characteristics. Typically, most parts video frames in a video sequence vary from frame-to-frame or at least within a certain number of frames. However, some parts of video frames remain the same for a certain number of frames or an entire video sequence. The parts of video frames that vary are presumed to include video that was originally captured by a video camera or similar input device. Freeze frame and data mismatch faults in a video sequence are undesirable. The parts of video frames that remain the same are presumed to include image overlays that may provide text information, icons, symbols, or test patterns. Freeze frame faults occur when the same video frame is repeated by the video processing system instead of properly continuing with subsequent video frames of the video sequence. Data mismatch faults occur when the expected content of an image overlay is not present in a given video frame. Current solutions for fault detection in video processing systems insert undesirable latency in the video processing, increase the memory bandwidth required for video processing, and increase power consumption for the video processing system. Improved fault detection techniques are needed.